A Large-Scale Linear Regression Sentiment Model

This report details the findings in building a large-scale linear regression sentiment model for an Amazon book review corpus. We studied and applied a number of regression and NLP techniques, including Unigram/Bigrams, stop-word removal, ridge and lasso regression. 1. DATA CORPUS The corpus contains 975194 non-distinct book reviews collected from Amazon by Mark Dredze and others at Johns Hopkins. Along side the textual reviews we have numerically scored sentiment data on a scale of 1-5. The goal is to learn a linear regression based sentiment model from the textual reviews which can be applied to future reviews to obtain an estimate of the numerical sentiment score. 2. DATA PROCESSING The original data was represented in XML format. A simple tokenizer had been applied to obtain a flattened representation consisting of a reverse-indexed dictionary matrix, which allowed review-text and sentiment scores can be extracted. We obtained document boundaries by segmenting at token positions corresponding to the pair. The textual body of a review is extracted by matching token positions for the pair. Review titles often contain sentiment charged summaries of the review and hence are extracted and concatenated with the review body. Finally, numerical ratings are obtained by matching token positions bewtween the pair. 2.1 Duplicate removal It turns out that many of the reviews in the corpus were duplicates. to remove duplicates, we computed a textual-hash of the first 20 words (or the maximum number of words in the textual-review if less than 20 words) and discarded reviews for which the hash matched a prior review. At the end of this process we obtained 494761 distinct reviews, roughly half the size of the original corpus. 2.2 Featurization We experimented with both bag-of-words and bag-of-bigrams models. Each review is featurized into a multinominal count of words and bigrams. These counts are then normailized via tf-idf scores, computed as: St,d = tft,d × log2 N